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Abstract 

We investigate the logical depth function of finite strings. For the function associated 
with string x with argument b and value d we have that d is the least running time 
of the computation of an element of the set of b- incompressible programs for x. For 
a given string we consider the possible gap between two values of this function if 
the arguments differ by just a constant. We show that there is an infinite sequence 
of strings, every successor one bit longer than its predecessor, such that the widths 
of the associated gaps rise at least as fast as the Busy Beaver function, that is, 
faster than any computable function. As a consequence, the computation time of 
shortest programs associated with this sequence of strings plus programs that are a 
certain number of bits longer rises (like the Busy Beaver function) faster than any 
computable function. 
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1 Introduction 

To compute a string x from a shortest program for it may take a very long 
time. However, to compute the same string from a program of about length \x\ 
that essentially spells out x takes very little time, since we can simply copy. 
A program for x of larger length than a given program for x may decrease the 
computation time but certainly does not increase it. Therefore, the longest 
computation time is associated with a shortest program for x. Such a program 
is incompressible. There arises the question how much time can be saved by 
computing a given string from a 6-incompressible program (a program that 
can be compressed by at most b bits) when b rises. 

1.1 Related Work 

The minimum time to compute a string by a ^-incompressible program was 
first considered in [?] . This minimum time is called the logical depth at signif- 
icance b of the string concerned. Definitions, variations, discussion and early 
results can be found in the given reference (but see Remark [T]) . A more formal 
treatment as well as an intuitive approach was given in the textbook [?], Sec- 
tion 7.7. In [?] the notion of computational depth is defined as K l (x) — K{x) 
(see definitions below). This is similar to the negative logarithm of the ex- 
pression Qfj(x) /Qu(x) in Definition [3], since [?] proved in the so-called Coding 
Theorem that — log Qu(x) = K(x) up to a constant additive term but requires 
also — logQulx) = K f (x) up to a small (preferable constant) additive term. 
The last equality is a major open problem in Kolmogorov complexity theory, 
see [?] Exercises 7.6.3 and 7.6.4. 

1.2 Results 

We prove that there is an infinite sequence of strings, each string one bit 
longer than its predecessor, such that for each string in the sequence the 
difference in logical depths associated with a fixed constant difference in sig- 
nificance increases exponentially (Theorem [3]). We prove next (Theorem H] 
and Corollary [2]) for (possibly another) infinite sequence of strings that this 
exponential increase can be, in fact, an increase as fast as the Busy Beaver 
function, the first incomputable function [?]. In fact, this function is total and 
rises faster than any computable function. Consequently, the running times of 
shortest programs associated with this sequence and certain longer programs 
rises faster than any computable function (Corollary [3]). Therefore the func- 
tion such that its nth value is equal to the logical depth of the nth string in 
the sequence at appropriate low fixed constant significance, is incomputable. 
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The rest of the paper is organized as follows: in Section [2j we introduce some 
notation, definitions and basic results needed for the paper. In Section [3j we 
prove the results mentioned. 



2 Preliminaries 

We use string or program to mean a finite binary string. The alphabet is 
S = {0, 1}, and S* = {0, 1}* is the set of all strings. Strings are denoted by 
the letters x, y and z. The length of a string x (the number of occurrences 
of bits in it) is denoted by and the empty string by e. Thus, |e| = 0. We 
use the notation S n for the set of strings of length n. We also use the binary 
logarithm which is denoted by "log." 

Often the resource-bounds in this paper are time constructible. There are many 
definitions. For example, there is a Turing machine whose running time is 
exactly t(n) on every input of size n, for some function t : N — > N, where 
N is the set of natural numbers. However, in this paper there are also many 
functions that are not time constructible. An example is the Busy Beaver 
function BB : N — > N (Definition [7]) which is not computable (it rises faster 
than any computable function). In this case, and when we just mean a number 
of steps, we indicate in the superscript the number of steps taken, usually 
using d. Given a program p, we denote its running time by time(p). Given two 
functions / and g, we say that / G 0(g) if there is a constant c > 0, such that 
f(n) < c ■ g(n), for all but finitely many nGN. 

2.1 Kolmogorov Complexity 

We refer the reader to the textbook [?] for details, notions, and history. We use 
Turing machines with a read-only one-way input tape, one or more (a finite 
number) of work tapes at which the computation takes place, and a one- 
way write-only output tape. All tapes are semi-infinite, divided into squares, 
and each square can contain a symbol from a given alphabet or blanks. The 
machine uses for all of its tapes a finite alphabet and all tapes are one-way 
infinite. Initially, the input tape is inscribed with a semi-infinite sequence of 
0's and l's. The other tapes are empty (contain only blanks). At the start, 
all tape heads scan the leftmost square on their tape. If the machine halts 
for a certain input then the contents of the scanned segment of input tape 
is called the program or input, and the contents of the output tape is called 
the output. The machine thus described is a prefix Turing machine. Denote it 
by T. If T terminates, then the program is p and the output is T(p). The set 
V = {p : T(p) < oo} is prefix- free (no element of the set is a proper prefix of 
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another element). By the ubiquitous Kraft inequality [?] we have 

]T2- |P| <1. (1) 

The same holds for a fixed conditional or auxiliary. The above unconditional 
case corresponds to the case where the conditional is e. Among the univer- 
sal prefix-free Turing machines we consider a special subclass called optimal, 
see Definition 2.0.1 in [?]. To illustrate this concept: let T\,T2, ... be a stan- 
dard enumeration of (prefix) Turing machines, and let U\ be one of them. If 
Ui(i, pp) = Ti{p) for every index i and program p and outputs for inputs that 
are not of the form pp (doubling of p), then U\ is also universal. However, U\ 
can not be used to define Kolmogorov complexity. For that we need a machine 
Ui such that U2{i,p) = Ti(p) for every i,p. A machine such as U2 is called an 
optimal prefix Turing machine. Optimal (prefix) Turing machines are a strict 
subclass of universal (prefix) Turing machines. The above example illustrates 
the strictness. To define Kolmogorov complexity we require optimal (prefix) 
Turing machines and not just universal (prefix) Turing machines. The term 
'optimal' comes from the founding paper [?]. 

Definition 1 Let U be an optimal prefix-free Turing machine, and x, y be 
strings. The prefix-free Kolmogorov complexity K(x\y) of x given y is defined 
by 

K(x\y) = min{|p| : U(p,y) = x}. 

Let t : N — > N be a time-constructible function. The notation U t (p,y) = x 
means that U{p,y) = x within t(\x\) steps. The t-time-bounded prefix-free 
Kolmogorov complexity K l (x\y) of x given y is defined by 

K l (x\y) = min{|p| : U l (p,y) = x}. 

The default value for the auxiliary input y for the program p, is the empty 
string e. To avoid overloaded notation we usually drop this argument in case it 
is there. A well-known result of [?] states that n steps of a multiworktape (pre- 
fix) Turing machine can be simulated in O(nlogn) steps of a two-worktape 
(prefix) Turing machine. Thus, we can choose a reference optimal Turing ma- 
chine U such that U(i,p) = Ti(p) for all %,p. If Ti(p) terminates in time t(n) 
then U(i,p) terminates in time 0(t(n) log t(n)). Let a; be a string. Denote by 
x* the first shortest program in standard enumeration such that U(x*) = x. 

Definition 2 A string x is c-incompressible if 

K{x) > \x\ — c. 

A simple counting argument can show the existence of c-incompressible strings 
of every length for the plain complexity C(x). Since K(x) > C(x) we have the 
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following: 



Theorem 1 There are at least 2 n (l - 2 c ) + 1 strings x e S" that are c- 
incompressible. 

The definition of logical depth [?] is based on Qu(x), the so-called a priori 
probability [?] and its time bounded version. Here U d (p) means that U(p) 
terminates in at most al steps. 

Qu(x)= £ 2-W Q d u (x)= £ 2-1*1 

U(p)=x U d (p)=x 

We give two tentative definitions. 

Definition 3 The logical depth, tentative version 1, of a string x at signifi- 
cance level e = 2~ b is 

depth^ix) = min [d : > e) 

d 1 Q v (x) ~ J 



Using a program that is longer than another program for output x can shorten 
the computation time. Thus, the 6-significant logical depth of an object x is 
defined as the minimal time the reference Turing machine needs to compute x 
by a 6-incompressibe program (one that can be compressed by at most b bits). 

Definition 4 Let x be a string, b a nonnegative integer. A string's logical 
depth, tentative version 2, at significance level b, is: 

(2) 

depth\ \x) = mm{time(p) : \p\ < K(p) + b AU(p) = x} . 



Version (2) is stronger than version (1) in that in the version (2) every in- 
dividual program at significance level b must take at most depth[ 2 ^(x) steps 
to compute x, while version (1) requires only that a weighted average of all 
programs for x require at most depth^^x) steps. The quantitative difference 
is small as the following theorem shows (a combination of Theorem 7.7.1 and 
Exercise 7.7.1 in [?]). 

Remark 1 The originating reference [?] can be used for inspiration but is not 
everywhere trustworthy. For example on page 245 first paragraph it states that 
"Given a string x, its length n and a value of the significance parameter s . . . 
one can compute the depth." This is false since Definition [3] is incomputable 
since Qu{%) is incomputable because of the halting problem, and Definition H] 
is incomputable because K(p) is incomputable. O 

Theorem 2 A string x satisfies d = depthfl b (x) (b up to precision mm{K(d), K(b)}+ 
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0(1) ) if and only if d is the least number of steps needed by a b-incompressible 
program to print x. 

In [?] the precise form of the theorem is given as 

2 b+mm{K(d),K(b)}+0(l) ~ Q v (x) ~ 2 b +°W ' 

in the sense that the proof of right inequality shows the "if" part in the above 
statement of Theorem [2] while the proof of the left inequality shows the "only 
if" part. 

Notice that by the Coding Theorem of [?] which states — log Qu(x) = K(x) + 
0(1) we have: 

2 -K(x)+0{l) l 

2b+min{K(d),K(b)}+0(l) ~ 2 K ( x )+ min -{ K ( d ), K ( b )}+ b +°( 1 ) 

2 -K(x)+0(l) i 

< O d (r) < - = 

_ vt/w _ 2 b +o(i) 2 K{ - x ) +b+0 W 

Theorem [2] shows that the quantitative difference between the two versions of 
logical depth are small. We choose version 2 as our final definition of logical 
depth. 

Definition 5 A string x is (d,b)-deep if Definition @] holds. 

This means that the logical depth of a string x can for all purposes can be 
considered as a function. As a function it is easier to interpret. 

Definition 6 The function f x : N ->■ N defined by f x (b) = d, where d is the 
least number of steps needed by a 6-incompressible program to print x, is the 
functional logical depth of x. 

Remark 2 It is easy to see that f x (0) is the least number of steps to com- 
pute x from an incompressible program. For example, x* is known to be in- 
compressible [?]. Thus, time(x*) > f x {0)- (As far as the authors know it is 
not known that if U(p) = x and p is incompressible then p is a shortest 
program for x.) For higher arguments f x is monotonic nonincreasing until 
f x (\x\ — K(x) + 0(1)) = 0(\x\ log |x|), the 0(1) term represents a program to 
copy the literal representation of x in 0(\x\ log |x|) steps. O 

It is the aim of this paper to study the properties the graph of f x can have. 
For example, if x is random (i.e., |x| = n and K(x) > n + K(n)) then always 
b = 0(1) and always d = O(nlogn). These x's, but not only these, are called 
shallow. 
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To make comparisons between the logical depths of two arguments feasible, 
one can scale the running times for different programs. Here we use, as scaling 
factor, a Busy Beaver function as was first done in [?]. 

Definition 7 The Busy Beaver function BB : N — >■ N is defined by 

BB(n) = max {running time of U(p) < 00} 

p:\p\<n 

Definition 8 The Busy Beaver logical depth, at significance level b, of a 
string x is defined by 

depth BB (x) = min {I : \p\ < K{p) + b and U(p) = x in time at most BB(l)} . 

The Busy Beaver functional logical depth, at significance level b, of a string x 
is defined by 

f BB {b) = min {I : \p\ < K(p) + b and U(p) = x in time at most BB(l)} . 

If b is large enough so that we can have p = qx* with \q\ < b and q a copy 
program running in time polynomial in \x\, then there exists ap as used in this 
definition since BB{1) — > 00 with growing I. Notice that Definition [8] rescales 
the logical depth of Definition HJ since BB' 1 (depth b (x)) = depthf B (x). From 
Definition E it follows directly that K(depthf B (x)) < K(x,b) + 0(1). To see 
this, one can simulate any computation that terminates, keeping track of the 
number of computation steps and thus, K(time(p)) < K(p) + 0(1) which 
implies the inequality. 



3 The graph of logical depth 

Even slight changes of the significance level b can cause large changes in logical 
depth. 

Theorem 3 Let c, k± and hi be appropriate constants. For each large enough 
n there is a string x of length n such that depth k2 (x) > 2 n and depth 2c+kl (x) = 
0(n log n). (In terms of the functional logical depth f x (hi) > 2 n and f x (2c + 
ki) = 0(n logn) ). 

Proof. Assume the conditions in the theorem. Consider the set 

A = {x E S n : \p\ < n - c A U(p) = x in at most 2 n steps }. 

The set A is a subset of the c-compressible strings. Let B = S n \ A. Then B 
contains all c-incompressible strings, and therefore \B\ > 2 n (l — 2~ c ). (Note 
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that B also contains c-compressible strings, for instance y such that U(y*) = y 
in more than 2 n steps and \y*\ < n — c.) Let x G B be a (c + /^-incompressible 
string such that for all strings p with U(p) = x and \p\ < n + O(logn) holds 

n — c — k\ < K{p) < n — c — k 2 , (2) 

where < k 2 < &i • We defer the proof that such strings x exist to Lemma [TJ 
Assume for now that these strings exist. 

Claim 3 depth k2 (x) > 2 n . 

Proof. Assume that depth fe2 (x) < 2 n . By Definition of logical depth, there 
exists a /^-incompressible p with U{p) = x in time less then 2 n such that (T5]) 
holds. Therefore 

K{p) + k 2 < n — c — k 2 + k 2 = n — c. 
Hence x G A: contradiction. □ 

Claim 4 depth 2c+kl (x) = O(nlogn). 

Proof. Let c be the length of a shortest prefix-free version of the program 
print. Then print(x) with x literal has length at most n + c. By Definition [5] 
of logical depth there exist (2c + /^-incompressible programs p with U(p) = x 
such that by ([2]) we have: 

K(p) + 2c + k x > n - c - k x + 2c + k x = n + c. 

Let print (x) be (2c + /^-incompressible. That is, x is (c + /^-incompressible. 
Then depth 2c+fci (x) is at most the running time of print(x), which is at most 
0(n\ogn). □ □ 

Lemma 1 Let c,ki,k 2 > be large enough constants < k 2 < k\, n a large 
enough positive integer, and B the set described in the last proof. There exists 
a (c+Zci) -incompressible string x G B such that for all strings p with U{p) = x 
and \p\ = n + O(logn) inequality (j2J) holds. 

Proof. Let a, b > be constants to be chosen later, n large enough, and q be 
the lexicographic first program (not necessarily for x) of length n — c — a with 
the longest running time of all programs of these lengths. The longest running 
time of a a string is defined as the maximal running time of a prefix of the 
string for which the computation halts. Therefore q has the longest running 
time of all programs of length at most n — c — a. To see this, suppose there is 
a program q' with \q'\ < n — c — a with a longer running time than q. Then 
we can pad q' with nonsignificant O's to the length of q. We write this longest 
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running time of q (the maximum of the running times of the halting prefixes 
of q) as time'(q). 

Let x with \x\ = n be the first string in lexicographic order such that every 
program p with U(p) = x and \p\ < n + O(logn) satisfies 

K time ' (q \p)>n-c~a. (3) 

(Such x exist. For example, if \y\ = n and K(y) > n — c — a then with r 
instead of p is satisfied for all strings r such that U(r) = y.) For the above 
Definition [3] we have for all such p 

K(p) < \q\ + b < n — c — a + b 

for a constant b > 0. Namely, from g we can compute time'(q) and n — c — a 
(since it is equal to |g|), from this we can compute the set of p's satisfying 
OH])- Since we are only interested in at most n + O(logn) length p's the set 
of relevant p's is finite. By scrutinizing this finite set and knowing n through 
the constant c — a while we knew n — c — a we can determine the string x. 
All this can be done (including knowing the constant c — a) using a 6-length 
program with b > a constant. Moreover, we have for every p with U(p) = x 
and \p\ < n + O(logn) that 

K(p) > n — c — a. 

To see this, fix p. To the contrary assume K(p) < n — c — a. The construction 
of q and K(p) < n — c — a imply time(p*) < time'(q). Therefore, K time ^ p *\p){= 
p) = K time '( q \p) < n — c — a contradicting Q. Since we can choose p = x* 
this also proves that x of length n is (c — a)-incompressible. 

Set a > b and ki = a and k<i = a — b. This proves the lemma. □ 

Theorem 4 Letc, k\, andk 2 be appropriate constants. For every large enough 
n there is a string x of length n such that depth^f ''(x) > n and depth^+^ix) = 
O(logn). (In terms of the functional logical depth f^ B {2c+ki) = O(logn) and 
L BB (h) > n) 

Proof. The proof of Theorem [3] gives the intuition to understand the idea of 
the present proof. Consider 

A = {x G S n : \p\ < n — c A U{p) = x in at most BB{n) steps}. 

Taking B = T, n \A, we know that B has at least 2 n (l— 2~ c ) elements. Let x G B 
be a (c + /^-incompressible string such that for all strings p with U(p) = x 
and \p\ < n + O(logn) inequality ([2]) holds. Such x exist by Lemma [TJ 

Claim 5 depth B f(x) > n. 
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Proof. Assume that depth fc2 (x) < n. Then, by definition of Busy Beaver 
logical depth there exists a ^-incompressible p with U(p) = x such that by 
([2]) we have 

K(p) + k 2 < n — c — k 2 + k 2 = n — c 
such that U(p) = x in time less than BB(n). Hence x G A: contradiction. □ 

Claim 6 depth^^_ k (x) < O(logn). 

Proof. Let c be the length of a shortest prefix-free version of the program 
print. Then print (x) with x literal has length at most n+c. By Definition[5]of 
logical depth there exists a (2c + /^-incompressible program p with U(p) = x 
such that by (j2J) we have 

K(p) + 2c + ki>n-c-k 1 + 2c + ki = n + c. 

Let print (a;) be (2c+/ci)-incompressible. Then, the running time depth 2c+fei (x) 
is at most 55 _1 (time(print(x))) = BB~ l (n\ogn) = O(logn) (since the 
Busy Beaver function grows faster than any computable function). □ □ 

We can adapt the argument presented above to prove that if we set 

A = {x G S n : \p\ < n — g(n) A U{jp) — x in at most BB(n) steps}, 

where g(n) is a computable function c < g(n) < n — logn then the arguments 
go through as well. The above case corresponds to g{n) = c. If we set g{n) = 
log n for instance then we obtain the result below. 

Corollary 1 Let c, k\ and k 2 be sufficiently large constants. For every suffi- 
ciently large n there is a string x of length n such that we have depthf 2 B (x) > n 
and depth^ 21ogn (x) = O(logn). 

Proof. (Sketch) The proof is equal to the previous one with the following 
adaptations: 

A = {x G E n : \p\ < n — logn AU(p) = x in at most BB{n) steps.} 

The complement set B has at least \B\ > 2 n (l — 2~ logn ) = 2 n — 2 n /n elements. 
With similar reasoning as Lemma [T] we can show the existence of a string in 
the complement of A satisfying n — k\ — logn < K[x) < n — k 2 — logn. □ 

Corollary 2 Let s and / be large enough constants with s < f. (Here 
s corresponds to k 2 above and / corresponds to 2c + k\.) There exists an 
infinite sequence of strings Xi,x 2 ,... with \x i+ i\ = \xi\ + 1, such that for 
all programs p\ with \p\\ < K{p\) + s with U(p\) = Xi in d\ steps and all 
programs pf with \pf\ < K{pf) + f with U(pf) = Xi in df steps we have 



10 



df — d\ > BB(i — 1) — BB (log 2 i) (i > 1). Hence the gap in logical depths with 
a fixed constant difference in significance of strings Xi in this sequence rises 
at least as BB(i — 1) — BB (log 2 i). This gap rises faster than any computable 
function since BB(i - l)/2 does and BB(i - l)/2 > ££(« - 1) - BB(\og 2 i) 
except possibly for i very small. 

Corollary 3 For every string x by definition f x (0) > time(x*) since x* is 
a O-incompressible string. Moreover, the function / x is monotonic nonincreas- 
ing for all strings x. By the Corollary [2] there is a sequence xi,X2, ■ ■ ■ such 
that the values of f Xj for certain arguments differ at least by BBii — l)/2. 
Therefore, certainly the values of f Xi {0) > BBii — 1). Hence, there exists 
an infinite sequence of strings xi,x 2 , ■ ■ ■ with = \xi\ + 1, such that the 

associated sequence of shortest programs x\,x 2 ,--- and programs of length 
+ k, \x 2 \ + k, . . . (0 < k < k 2 ) have a computation times in excess of 
BB(0)/2, BB(l)/2, . . . (except for possibly a small initial segment). Hence 
the times to compute Xj from x*, and from programs up to \x*\ + k 2 , is in 
excess of BB(i — l)/2 (i = 1, 2, . . .), a function that is incomputable and rises 
faster than any computable function for i is not too small. 

Corollary 4 There is nothing that we used from the Busy Beaver function 
for the result in Theorem H] and Corollaries [T] through [3] except for the fact that 
it is well-defined and details about the print instruction. The same results hold 
for other functions that are well-defined as the Busy Beaver function and grow 
still faster. The proof is similar. For example functions hi(n) = BB{n) BB ^ 
and 



an exponential stack of 5i?(ra)'s of height BB(n). 



4 Conclusion 

We studied the behavior of the logical depth function associated with a string 
x of length n. This function is monotonic nonincreasing. For argument the 
logical depth is at most the running time of the computation from a shortest 
program x* to x. The function decreases to 0(n log n) for the argument \x\ — 
K(x). We show that there is an infinite sequence of strings such that the 
difference in logical depths for a fixed constant difference in arguments rises 
faster than any computable function, that is, incomputably fast as the Busy 
Beaver function. This shows that logical depth can increase tremendously for 
only a constant difference in arguments. Moreover, there is an infinite sequence 
of strings such that the minimal computation times of associated shortest 
programs and even shortest programs plus a certain number of bits rises at 
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least as fast as the Busy Beaver function, that is, faster than any computable 
function. This seems a shot at the determining the possibly extreme running 
times of shortest programs and programs that are a certain number of bits 
longer. 
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